Camera and method of photographing good image

ABSTRACT

A camera includes a release button, an input unit, an A/D converter, a memory, a control unit, an alarm, a recording unit and an output unit. The memory stores data for the image converted by the A/D converter. The control unit judges whether or not the image stored in the memory satisfies a predetermined photographing condition and outputs a timing signal when the image satisfies the photographing condition. The alarm outputs an alarm signal to a photographer. The recording unit records the refined image on a recording medium. The output unit outputs the refined image.

[0001] This application is a Divisional of co-pending application Ser.No. 09/586,600, filed on Jun. 2, 2000, the entire contents of which arehereby incorporated by reference and for which priority is claimed under35 U.S.C. § 120; this application also claims priority based on aJapanese patent applications, Hei 11-157159 filed on Jun. 3, 1999, andHei 11-158666 filed on Jun. 4, 1999, the contents of which areincorporated herein by reference.

BACKGROUND OF THE INVENTION

[0002] 1. Field of the Invention

[0003] The present invention relates to a camera, and more particularlyto a camera capable of automatically photographing an image of a subjectwhen the subject satisfies a predetermined photographing condition.

[0004] 2. Description of the Related Art

[0005] Conventionally, a technique is known to correct a photograph sothat a person photographed by a camera can be satisfied with the result.However, this technique requires a high degree of skill. Furthermore, itis difficult to correct a person's face in the photograph when he or sheis blinking or is not smiling, to a face as if he or she is not blinkingor is smiling.

[0006] On the other hand, Japanese Patent Laid-open Publication (Kokai)H9-212620 and Japanese Patent Laid-open Publication (Kokai) H10-191216disclose a technique to continuously photograph a plurality of images.Those images are displayed, and the person photographed by the cameracan select a desirable image from among those images.

[0007] Japanese Patent Laid-open Publication (Kokai) H5-40303, H4-156526and H5-100148 disclose cameras which can automatically judge the timingfor photographing images.

[0008] However, this was troublesome because the photographed person orthe photographer needed to select the desired image by checking all ofthe images. Furthermore, when a lot of people are photographed in theimage, it is more difficult to select an image that all of them aresatisfied with.

[0009] Furthermore, images are photographed at the timing when thephotographer judges it is the best timing. Therefore, the photographerstiming is not always matched with the best timing for the photographedperson. In addition, when a lot of people are photographed in the image,it is more difficult to judge the best timing at which many of them willbe satisfied with the image.

SUMMARY OF THE INVENTION

[0010] Therefore, it is an object of the present invention to provide acamera which overcomes the above issues in the related art. This objectis achieved by combinations described in the independent claims. Thedependent claims define further advantageous and exemplary combinationsof the present invention.

[0011] According to the first aspect of the present invention, a cameracomprises: an image data input unit forming an image of a subject forphotographing said subject; a condition storing unit storing apredetermined photographing condition related to a desirable subject;and a timing signal generator outputting a timing signal when saidsubject satisfies said photographing condition.

[0012] The camera may include an extractor extracting data of an aimedobject from said image of said subject based on an extracting condition,wherein said photographing condition may include a predeterminedcondition related to a desirable aimed object and said timing signalgenerator outputs said timing signal when said aimed object satisfiessaid photographing condition.

[0013] The extracting condition may be based on depth information ofsaid image indicating the distance to each part of said subject.

[0014] The extractor may detect data of a judgement location from saiddata of said aimed object in said image based on a detecting conditiondifferent from said extracting condition, said photographing conditionmay include a predetermined photographing condition related to adesirable judgement location, and the timing signal generator may outputsaid timing signal when said judgement location satisfies saidphotographing condition.

[0015] The extractor may extract data of a plurality of said aimedobjects from said image; and said timing signal generator may outputsaid timing signal when said plurality of aimed objects satisfy saidphotographing condition.

[0016] The timing signal generator may output said timing signal whenthe ratio of said aimed objects satisfying said photographing conditionagainst all of said plurality of said aimed object exceeds apredetermined ratio.

[0017] The extractor may detect data of a plurality of judgementlocations from each of said data of said plurality of aimed objectsbased on a detecting condition different from said first condition, saidphotographing condition may include a predetermined photographingcondition related to said judgement location, and said timing signalgenerator may output said timing signal when said plurality of saidjudgement locations satisfy said photographing condition.

[0018] The timing signal generator may output said timing signal whenthe ratio of said judgement locations satisfying said photographingcondition against all of said plurality of said aimed object exceeds apredetermined ratio.

[0019] The camera may include an image-pickup control unit controllingsaid input unit for photographing said image based on said timingsignal.

[0020] The camera may include an illuminator illuminating said subjectbased on said timing signal.

[0021] The camera may include a recording unit recording said image on areplaceable nonvolatile recording medium based on said timing signal.

[0022] The camera may include an alarm outputting an alarm signal fornotifying that said subject satisfies said photographing condition basedon said timing signal.

[0023] The photographing condition may include a plurality ofphotographing conditions, and said camera may include acondition-setting unit previously selecting at least one of saidphotographing conditions, for photographing said image, from among saidplurality of photographing conditions.

[0024] The camera may include: an input condition determining unitdetermining an input condition for inputting said image based oninformation of said judgement location detected by said extractor; andan image-forming control unit controlling an input unit for forming saidimage of said subject based on said input condition.

[0025] The camera as set forth in claim may include an image processingunit processing said image based on information of said judgementlocation detected by said extractor.

[0026] According to the first aspect of the present invention, a cameracomprises: an image data input unit forming a plurality of images of asubject for photographing said subject; a condition storing unit storinga predetermined photographing condition related to a desirable variationof said subject; a variation detector detecting variation of saidsubject in said plurality of said images based on information of saidplurality of images; and a timing signal generator outputting a timingsignal when said variation of said subject satisfies said photographingcondition.

[0027] The camera may include: an extractor extracting data of an aimedobject from each of said plurality of images of said subject based on anextracting condition, wherein said photographing condition may include apredetermined condition related to a desirable aimed object, saidvariation detector may detect variation of said aimed object in saidplurality of images based on said information of said plurality ofimages, and said timing signal generator may output said timing signalwhen said variation of said aimed object satisfies said photographingcondition.

[0028] The extracting condition may be based on depth information ofsaid plurality of images indicating the distance to each part of saidsubject.

[0029] The extractor may detect data of a judgement location from saiddata of said aimed object in each of said plurality of images based on adetecting condition different from said extracting condition, saidphotographing condition may include a predetermined photographingcondition related to a desirable judgement location, said variationdetector may detect variation of said judgement location in saidplurality of images based on said information of said plurality ofimages, and said timing signal generator may output said timing signalwhen said variation of said judgement location satisfies saidphotographing condition.

[0030] The photographing condition may include a predetermined startingcondition for starting detection of said variation of said judgementlocation, and said variation detector may start detecting said variationof said judgement location when said judgement location satisfies saidstarting condition.

[0031] The extractor may extract data of a plurality of said aimedobjects from each of said plurality of images, said variation detectormay detect variation of each of said plurality of said aimed objects insaid plurality of images based on information of said plurality ofimages, and said timing signal generator may output said timing signalwhen said variation of said plurality of said aimed objects satisfy saidphotographing condition.

[0032] The extractor may detect data of a plurality of judgementlocations from each of said data of said plurality of aimed objectsbased on a detecting condition different from said extracting condition,said photographing condition may include a predetermined photographingcondition related to desirable variation of said judgement location,said variation detector may detect variation of each of said pluralityof said judgement locations in said plurality of images based oninformation of said plurality of images, and said timing signalgenerator may output said timing signal when said variation of saidplurality of said judgement locations satisfy said photographingcondition.

[0033] The camera may include an image pickup control unit controllingsaid input unit for photographing said image based on said timingsignal.

[0034] The camera may include an illuminator illuminating said subjectbased on said timing signal.

[0035] The camera may include a recording unit recording said image on areplaceable nonvolatile recording medium based on said timing signal.

[0036] The camera may include an alarm outputting an alarm signal fornotifying that said subject satisfies said photographing condition basedon said timing signal.

[0037] The photographing condition may include a plurality ofphotographing conditions, and said camera may include acondition-setting unit previously selecting at least one of saidphotographing conditions for photographing said image, from among saidplurality of photographing conditions.

[0038] The timing signal generator may select said judgement locationsatisfying said photographing condition from among said plurality ofsaid judgement locations in said plurality of images, and outputsinformation for said aimed object including said judgement location, andthe camera may include: an input condition determining unit determiningan input condition for inputting said image based on information forsaid judgement location; and an image forming control unit controllingan input unit for forming said image of said subject based on said inputcondition.

[0039] The timing signal generator may select said judgement locationsatisfying said photographing condition from among said plurality ofsaid judgement locations in said plurality of images, and outputsinformation for said aimed object including said judgement location, andsaid camera may include an image processing unit processing said imagebased on said information for said judgement location.

[0040] According to the third aspect of the present invention, a methodof photographing an image of a subject comprises outputting a timingsignal when said subject satisfies a predetermined photographingcondition.

[0041] The method may include: extracting data of an aimed object fromsaid image of said subject based on an extracting condition, whereinsaid photographing condition may include a predetermined conditionrelated to a desirable aimed object, and said timing signal may beoutput when said aimed object satisfies said photographing condition.

[0042] The extracting may include detecting data of a judgement locationfrom said data of said aimed object in said image based on a detectingcondition different from said extracting condition, said photographingcondition may include a predetermined photographing condition related toa desirable judgement location, and said timing signal may be outputwhen said judgement location satisfies said photographing condition.

[0043] The method may include photographing said subject based on saidtiming signal.

[0044] The method may include recording said photographed image of saidsubject on a replaceable nonvolatile recording medium based on saidtiming signal.

[0045] The method may include: determining an input condition forinputting said image based on information for said judgement locationdetected in said detecting step; and forming said image of said subjectbased on said input condition.

[0046] The method may include processing said image based on informationfor said judgement location detected in said detecting step.

[0047] According to the fourth aspect of the present invention, a methodof photographing a plurality of images of a subject comprising:detecting variation of said subject in said plurality of said imagesbased on information for said plurality of images; outputting a timingsignal when said variation of said subject satisfies a predeterminedphotographing condition related to a desirable variation of saidsubject.

[0048] The method may include extracting data of an aimed object fromeach of said plurality of images of said subject based on an extractingcondition, said detecting may include detecting variation of said aimedobject based on information for said image, and said timing signal maybe output when said variation of said aimed object satisfies saidphotographing condition.

[0049] The extraction of said aimed object may include detecting data ofa judgement location from said data of said aimed object in each of saidplurality of images based on a detecting condition different from saidextracting condition, said detecting variation of said subject mayinclude detecting variation of said judgement location based oninformation for said image, and said timing signal may be output whensaid variation of said judgement location satisfies said photographingcondition.

[0050] The photographing condition may include a predetermined startingcondition for starting detection of said variation of said judgementlocation, and said detecting of variation may start detecting saidvariation of said judgement location when said judgement locationsatisfies said starting condition.

[0051] The method may include photographing said image based on saidtiming signal.

[0052] This summary of the invention does not necessarily describe allnecessary features so that the invention may also be a sub-combinationof these described features.

BRIEF DESCRIPTION OF THE DRAWINGS

[0053]FIG. 1 shows a camera of the first embodiment according to thepresent invention,

[0054]FIG. 2 is a block diagram of the control unit of the firstembodiment,

[0055]FIG. 3 is a block diagram of the function of the extractor,

[0056]FIG. 4 is a flowchart showing a method of photographing an image,

[0057]FIG. 5 is a flowchart showing in detail the method of extracting aface part, step 106 in FIG. 4,

[0058]FIG. 6 is a flowchart showing in detail the method of detecting ajudgement location, step 108 in FIG. 4,

[0059]FIG. 7 is a flowchart showing in detail the method of generating atiming signal, step 110 in FIG. 4,

[0060]FIG. 8 is a flowchart showing in detail the method ofphotographing a refined image, step 112 in FIG. 4,

[0061]FIG. 9 is a flowchart showing in detail the method ofphotographing a refined image, step 112 in FIG. 4,

[0062]FIG. 10 is a flowchart showing in detail the method of generatinga timing signal, step 110 in FIG. 4,

[0063]FIG. 11 shows a camera of the second embodiment according to thepresent invention,

[0064]FIG. 12 is a block diagram of the control unit of the secondembodiment,

[0065]FIG. 13 is a block diagram of the control unit of the thirdembodiment,

[0066]FIG. 14 is a block diagram of the function of the extractor 60,

[0067]FIG. 15 is a block diagram of the function of the photographingcondition judging unit,

[0068]FIG. 16 is a flowchart showing in detail the method of detecting ajudgement location, step 108 in FIG. 4,

[0069]FIG. 17 is a flowchart showing in detail the method of generatinga timing signal, step 110 in FIG. 4,

[0070]FIG. 18 is a flowchart showing in detail the method ofphotographing a refined image, step 112 in FIG. 4,

[0071]FIG. 19 is a flowchart showing in detail the method of generatinga timing signal, step 110 in FIG. 4,

[0072]FIG. 20 is a block diagram of the control unit of the fourthembodiment,

[0073]FIG. 21 is a block diagram of the control unit of the fifthembodiment,

[0074]FIG. 22 is a flowchart showing a method of photographing an image,and

[0075]FIG. 23 shows a camera of the sixth embodiment.

DETAILED DESCRIPTION OF THE INVENTION

[0076] The invention will now be described based on the preferredembodiments, which do not intend to limit the scope of the presentinvention, but exemplify the invention. All of the features and thecombinations thereof described in the embodiment are not necessarilyessential to the invention.

[0077]FIG. 1 shows a camera 10 of the first embodiment according to thepresent invention. The camera 10 continuously photographs raw images ofa subject and determines the timing for photographing a refined imagebased on the previously photographed raw images. The camera 10photographs a refined image of the subject in accordance with the timingsignal. Therefore, the timing for photographing a refined image may beautomatically determined by the camera 10.

[0078] The camera 10 includes an input unit 20, an A/D converter 30, amemory 40, a control unit 50, a release button 52, an alarm 54, arecording unit 90 and an output unit 92. The camera 10 of thisembodiment further includes an illuminator 53. The camera 10 may be, forexample, a digital still camera or a digital video camera that canphotograph a still image.

[0079] The input unit 20 includes a parallactic image data input unit 22and a normal image data input unit 24. The parallactic image data inputunit 22 inputs parallactic images which are photographed from differentviewpoints. The parallactic image data input unit 22 has a parallacticlens 32, a parallactic shutter 34, and a parallactic charge coupleddevice (CCD) 36. The parallactic lens 32 forms an image of a subject.The parallactic shutter 34 has a plurality of shutter units each ofwhich serve as viewpoints. The parallactic shutter 34 opens one of theplurality of shutter units. The parallactic CCD 36 receives the image ofthe subject through the parallactic lens 32 and whichever of the shutterunits of the parallactic shutter 34 that are opened. The parallactic CCD36 also receives another image of the subject through the parallacticlens 32 and another of the shutter units of the parallactic shutter 34,which is opened at this time. The images received through theparallactic lens 32 and the parallactic shutter 34 form a parallacticimage. Thus, the parallactic CCD 36 receives the parallactic image ofthe subject formed by the parallactic lens 32 and converts it toelectronic signals.

[0080] The normal image data input unit 24 inputs a normal imagephotographed from a single viewpoint. The normal image data input unit24 has a lens 25, a lens stop 26, a shutter 27, a color filter 28 and acharge coupled device (CCD) 29. The lens 25 forms an image of a subject.The lens stop 26 adjusts an aperture condition. The shutter 27 adjustsexposure time. The color filter 28 separates RGB components of the lightreceived through the lens 25. The CCD 29 receives the image of thesubject formed by the lens 25 and converts it to electric signals.

[0081] The A/D converter 30 receives analog signals from the parallacticimage data input unit 22 and the normal image data input unit 24. TheA/D converter 30 converts the received analog signals to digital signalsand outputs the digital signals to the memory 40. The memory 40 storesthe input digital signals. This means that the memory 40 stores the dataof the parallactic image, the subject photographed by the parallacticimage data input unit 22, and the data of the normal image of thesubject photographed by the normal image data input unit 24.

[0082] The control unit 50 outputs a timing signal for startingphotographing of an image of a subject when the subject satisfies apredetermined photographic condition. The timing signal is input to theinput unit 20. The camera 10 then starts the photographing operationbased on the timing signal, to obtain a refined image of the subject.The control unit 50 processes the photographed refined image and outputsthe processed image. The control unit 50 controls at least one of thefollowing conditions: focus condition of the lens 25, aperture conditionof the lens stop 26, exposure time of the shutter 27, output signal ofthe CCD 29, condition of the parallactic shutter 34, and output signalof the parallactic CCD 36. The control unit 50 also controls theilluminator 53.

[0083] The release button 52 outputs to the control unit 50 a signal forstarting the photographing operation . This means that when a user ofthe camera 10 pushes the release button 52, the signal is output to thecontrol unit 50. The control unit 50 then controls the input unit 20 forphotographing an image of the subject.

[0084] As described above, the camera 10 is capable of automaticallyphotographing a refined image of the subject by determining best timingfor photographing the refined image. However, the camera 10 is alsocapable of photographing the image at a desirable timing for the user ofthe camera 10, when he/she pushes the release button 52. The camera 10may have a switch, not shown in the drawings, for selecting an automaticphotographing mode in which the best timing for photographing the imageis automatically determined, and a manual photographing mode in whichthe user of the camera 10 determines the desirable timing.

[0085] The alarm 54 outputs an alarm signal upon receiving the timingsignal from the control unit 50. The alarm 54 may be, for example, analarm generator or a light-emitting diode. Thus, the user of the camera10 can know the best timing determined by the camera 10 forphotographing a refined image of the subject.

[0086] The recording unit 90 records the image output from the controlunit 50 on a recording medium. The recording medium may be, for example,a magnetic recording medium such as a floppy disk, or a nonvolatilememory such as a flash memory.

[0087] The output unit 92 outputs the image recorded on the recordingmedium. The output unit 92 may be, for example, a printer or a monitor.The output unit 92 may be a small liquid crystal display (LCD) of thecamera 10. In this case, the user can see the image processed by thecontrol unit 50 immediately after photographing the image. The outputunit 92 may be an external monitor connected to the camera 10.

[0088]FIG. 2 is a block diagram of the control unit 50 according to thefirst embodiment. The control unit 50 includes an image pickup controlunit 56, an image forming control unit 58, an extractor 60, acondition-storing unit 70, a timing signal generator 80, an inputcondition determining unit 82, and an image processing unit 84.

[0089] The extractor 60 receives a parallactic image photographed by theparallactic image data input unit 22 and a raw image photographed by theimage data input unit 24, from the memory 40. The extractor 60 extractsan aimed object from the raw image based on the information obtainedfrom the parallactic image and the raw image. The information includesimage information of the raw image and depth information of theparallactic image. The aimed object defined here is an independentobject at which a photographer aims when photographing. The aimed objectmay be, for example, a person in a room when the person and the objectsin the room are photographed, a fish in an aquarium when the fish andthe aquarium are photographed, or a bird stopping on a branch of a treewhen the bird and the tree are photographed.

[0090] The extractor 60 then detects a judgement location from the aimedobject based on the information obtained from the parallactic images andthe raw images. The judgement location defined here is a location towhich specific attention is paid when selecting a desirable image. Thejudgement location may be, for example, an eye of a person when theperson is photographed, or a wing of a bird when the bird isphotographed. The aimed object may be an area including the judgementlocation, extracted for a certain purpose. The information for thejudgement location is output to the timing signal generator 80, theinput-condition-determining unit 82 and the image-processing unit 84.

[0091] The condition-storing unit 70 stores predetermined conditionsrelated to a judgement location which should be included in a raw imageobtained by photographing a subject. The best timing for photographing arefined image of the subject in this embodiment is when the aimed objectin the image is in good condition. This means that a judgement locationincluded in the aimed object satisfies the predetermined conditionsstored in the condition-storing unit 70. The condition-storing unit 70may store a plurality of photographing conditions. The condition-storingunit 70 may include a condition-setting unit, not shown in the drawings,by which a user can select at least one of the photographing conditionsfrom among a plurality of photographing conditions.

[0092] The timing signal generator 80 outputs a timing signal forphotographing an image. The timing signal generator 80 outputs thetiming signal when the judgement location detected by the extractor 60satisfies the predetermined photographing condition stored in thestoring unit 70.

[0093] The input-condition-determining unit 82 determines an inputcondition for inputting a refined image, based on the information forthe aimed object or the judgement location received from the extractor60. The input condition is output to the image-forming control unit 58.The input condition may be, for example, focus condition of the lens 25such that the aimed object including the judgement location is focussed.

[0094] As the input unit 20 inputs an image in accordance with the inputcondition such as the focus condition of the lens 25, determined by theinput-condition-determining unit 82, the camera 10 can photograph arefined image in which the subject is in good condition.

[0095] The image-forming control unit 58 controls the input unit 20 toform a refined image of the subject based on the input conditiondetermined by the input-condition-determining unit 82. This means thatthe image-forming control unit 58 controls at least one of theconditions including focus condition of the lens 25, aperture conditionof the lens stop 26, exposure time of the shutter 27, and condition ofthe parallactic shutter 34, based on the input condition.

[0096] The image-pickup control unit 56 controls the input unit 20 tophotograph a refined image of the subject based on the input conditiondetermined by the condition-determining unit 70. This means that theimage-pickup control unit 56 controls at least one of the conditionsincluding output signal of the CCD 29 and output signal of theparallactic CCD 36, based on the input condition. The output signal ofthe CCD 29 determines the gradation characteristics based on a gamma (γ)curve and sensitivity.

[0097] The image-pickup control unit 56 controls the input unit 20, tophotograph a refined image based on the timing signal output from thetiming signal generator 80. The image-pickup control unit 56 controlsthe image-processing unit 84 to process the refined image. Theimage-pickup control unit 56 may control the illuminator 53, forflashing a light preceding or at the same time as photographing arefined image by the input unit 20. The image-pickup control unit 56also controls the image-processing unit 84, to process the input refinedimage.

[0098] The image-processing unit 84 receives the refined imagephotographed by the image data input unit 24 from the memory 40. Theimage-processing unit 84 then processes the refined image based on theinformation for the aimed object or the judgement location extractedfrom the extractor 60.

[0099] Examples of the process condition for processing a normal imageare explained in the following.

[0100] The process condition for processing a normal image may relate tocompression of the image. The process condition in this case isdetermined based on the data for the aimed object. The image-processingunit 84 separately determines the compressing condition of the image forthe aimed object and for the components other than the aimed object sothat the quality of the aimed object does not deteriorate, even thoughthe data size of the image itself is compressed. The image processingunit 84 may separately determine the color compressing condition for theaimed object and the components other than the aimed object.

[0101] The process condition for processing a normal image may relate tocolor of the image. The process condition in this case is determinedbased on the depth information. The processing-condition-determiningunit 74 may, for example, separately determine the color condition forthe aimed object and the components other than the aimed object, so thatall the components have optimum gradation.

[0102] The image-processing unit 84 may determine a processing conditionin which the aimed object in the image is magnified and the magnifiedaimed object is composited with a background image. The background imagemay be the components included in the original image other than theaimed object, or an image previously selected by the user of the camera10. The image-processing unit 84 may then composite the data for theaimed object and the data for the components other than the aimed objectto form a composite image.

[0103] As described above, the extractor 60 extracts the data for theaimed object and the judgement location from the image, and the aimedobject and the judgement location can be processed separately from thecomponents other than these parts.

[0104] Since cameras are usually used to photograph human beings, thebest timing for photographing a refined image means that the targetedperson has a good appearance. The good appearance of the person may bewhen, for example, “the person is not blinking”, “the person's eyes arenot red-eyed”, “the person is looking at the camera”, or “the person issmiling”. The condition-storing unit 70 stores these conditions as thephotographing conditions. The condition-storing unit 70 may set aphotographing condition by selecting at least one of the photographingconditions stored therein.

[0105] The method of outputting a timing signal for photographing arefined image of a subject when a targeted person has a good appearancewill be explained. The condition-storing unit 70 stores conditions suchas “the person is not blinking”, “the person's eyes are not red-eyed”,“the person is looking at the camera”, and “the person is smiling” asthe photographing conditions. These photographing conditions relate tothe face of the person, and more specifically to the eyes or mouth ofthe person. Therefore, it is assumed in this embodiment that the aimedobject is the face area of the person and the judgement location is theeyes or mouth of the person.

[0106] Each of the photographing conditions has a reference situationfor the judgement location, which should meet the requirements of thephotographing condition. The condition-storing unit 70 also stores thereference situations for the judgement location, each respectivelycorresponding to each of the photographing conditions. The referencesituations for the judgement location corresponding to each of thephotographing conditions will be described in the following.

[0107] For the conditions such as “the person is not blinking”, “theperson's eyes are not red-eyed” and “the person is looking at thecamera”, the reference situation may relate to the shape of the eye,color of the eye, and size of the eye. For the condition such as “theperson is smiling”, the reference situation may also relate to the sizeof the eye, as well as shape of the mouth, and size of the mouth.Whether each of the judgement locations satisfies each of thesereference situations or not is judged in accordance with predeterminedalgorithms based on experience.

[0108] When the photographing condition “the person is not blinking” isselected, the judgement location may be the eye of the person. Thereference situation for the eye in this photographing condition will bedetermined as follows. When a person blinks, his/her eyelid hideshis/her eyeball. While he/she is blinking and his/her eye is partiallyclosed, the white part of his/her eyeball is especially hidden byhis/her eyelid. This means that when the person is not blinking, thewhite part of his/her eyeball should be relatively large. Therefore, thereference situation for the photographing condition “the person is notblinking” becomes “the white part of his/her eyeball has a largedimension”.

[0109] When the photographing condition “the person's eyes are notred-eyed” is selected, the judgement location may be the eyes of theperson. The reference situation for the eyes in this photographingcondition will be determined as follows. Eyes of a person are usuallyred-eyed when the person is photographed using a flash in a darksituation. This happens because the person's eyes cannot sensiblycompensate for the sudden brightness and his/her pupils become red. Thismeans that when the person's eyes look red-eyed, his/her pupils in eachiris become red and the rest of the iris does not become red. Typically,people of Asian descent have brown or dark brown colored irises, andpeople of European descent have green or blue colored irises. Therefore,the reference situation for the photographing condition “the person'seyes are not red-eyed” becomes “the red part in his/her iris has a smalldimension”.

[0110] When the photographing condition “the person is looking at thecamera” is selected, the judgement location may be the eye of theperson. The reference situation for the eye in this photographingcondition will be determined as follows. When a person is looking at thecamera, a line between the camera and the iris of the person and anormal vector of his/her iris are almost the same. Therefore, thereference situation for the photographing condition “the person islooking at the camera” becomes “the normal vector of the iris in his/hereye is approximately equal to the angle of the line between the cameraand his/her iris”.

[0111] When the photographing condition “the person is smiling” isselected, the judgement location may be the eyes and the mouth of theperson. The reference situation for the eyes and the mouth in thisphotographing condition will be determined as follows. When a person issmiling, although it depends on each person, his/her eyes becomerelatively thin. At this time, although it depends on each person,his/her mouth expands right-and-left wards and his/her teeth are shown.Therefore, the reference situations for the photographing condition “theperson is smiling” become “the white part in his/her eyes has a smalldimension”, “the width of his/her mouth is wide” and “the white area inhis/her mouth has a large dimension”.

[0112]FIG. 3 is a block diagram of the function of the extractor 60. Theextractor 60 includes a depth information extractor 62, an imageinformation extractor 64, an aimed object extractor 66 and a judgementlocation detector 68.

[0113] The depth information extractor 62 extracts the depth informationindicating the distance to each of components of the subject, based onthe data for the parallactic image received from the memory 40. Thismeans that the depth information extractor 62 determines a correspondingpoint for each of the components based on the parallactic image andgives a parallax amount. The depth information extractor 62 extracts thedepth information based on the parallax amount of each of thecomponents. Determining the corresponding point is a known technique,thus the explanation of this technique will be omitted. Extracting thedepth information based on the parallax amount is also a known techniqueusing the principle of triangulation, thus the explanation of thistechnique will be omitted.

[0114] The image information extractor 64 extracts the image informationfor normal images, from the data for the normal images received from thememory 40. The image information includes, for example, data for thenormal image such as luminescence distribution, intensity distribution,color distribution, texture distribution, and motion distribution.

[0115] The aimed object extractor 66 extracts data for the face area ofthe person as the aimed object, based on the depth information and theimage information. Each of the images may include, for example, aplurality of components. The aimed object extractor 66 recognizes eachof the components based on the depth information. The aimed objectextractor 66 then specifies the face area by referring to the depthinformation and the image information of each of the components. Themethod of specifying the face area will be described in the following.

[0116] The aimed object extractor 66 receives the photographingcondition from the condition-storing unit 70. The aimed object extractor66 extracts the aimed object based on the photographing condition. Inthis embodiment, the aimed object is the face of the photographedperson. Therefore, at first, the component including the face isspecified depending on assumptions such as “the person should be closeto the camera”, “the person should be in the middle of the image”, or“the proportional relationship of the height of the person to the widthand height of the image should be within a predetermined range”. Thedistance from the camera to each of the components in the image isevaluated based on the depth information. The distance from the centerof the image to each of the components in the image, and theproportional relationship of the height of the components are evaluatedbased on the image information. Each of the values is multiplied bypredetermined constants corresponding to each condition. The multipliedvalues are added for each of the components. The added values aredefined as weighted averages. The component having the largest weightedaverage is extracted as the component including the aimed object.

[0117] The constants by which the values for each of the components aremultiplied may be predetermined based on the aimed object. In thisembodiment, for example, the aimed object is assumed to be the face ofthe photographed person. Therefore, the aimed object extractor 66specifies the area having a skin color as the face part, based on theimage information. The colors of each of the components are evaluatedbased on the color distribution of the images. The values of the colordistribution may also be multiplied by predetermined constants and themultiplied values are added for each of the components to give theweighted averages.

[0118] As described above, the aimed object extractor 66 extracts anaimed object based on the depth information in addition to the imageinformation. Therefore, even when a plurality of people are photographedin the image and their faces are close to each other, the faces of thedifferent people can be distinctly extracted.

[0119] The judgement location detector 68 detects the judgement locationfrom the data for the face area extracted by the aimed object extractor66. The judgement location detector 68 receives the photographingcondition from the condition-storing unit 70. The judgement locationdetector 68 detects the judgement location based on the photographingcondition. In this embodiment, the judgement location is the eyes ormouth of the photographed person. Therefore, the judgement locationdetector 68 detects the eyes and mouth from the face area.

[0120] There is relatively little variation in the eyes of people withrespect to color, shapes or their place on the face. Therefore, patternsof eyes such as the color of the eyes, shape of the eyes, and the placeof the eyes on the face are previously determined, and the parts whichare approximately similar to the determined patterns of the eyes arerecognized as the judgement location on the face. Similarly, there isrelatively little variation in mouths of people with respect to color,shapes or place on the face. Therefore, patterns of the mouth are alsopreviously determined and the parts which are approximately similar tothe determined patterns of the mouth are recognized as the judgementlocation on the face.

[0121] The extractor 60 detects the judgement location from theextracted aimed object based on the image information for the aimedobject. Therefore, the extractor 60 does not extract locations havingsimilar shapes to the judgement location from the subject other than theaimed object included in the image.

[0122] The judgement location detector 68 then outputs the data for thedetected judgement locations to the timing signal generator 80.

[0123] Referring back to FIG. 2, the method for judging the best timingfor photographing an image will be explained in the following.

[0124] The timing signal generator 80 receives the data for the detectedjudgement locations from the extractor 60. The timing signal generator80 also receives the photographing condition from the condition-storingunit 70. The timing signal generator 80 compares each of the judgementlocations based on the reference situation for the photographingcondition. The timing signal generator 80 then generates a timing signalwhen the judgement location satisfies the reference situation for thephotographing condition.

[0125] When the photographing condition “the person is not blinking” isselected, the judgement location is the eyes and the reference situationis “the white part of his/her eyeball has a large dimension”, asdescribed above. Therefore, the timing signal generator 80 calculatesthe dimension of the white part of the eye detected by the judgementlocation detector 68 for each of the images, based on the imageinformation. The timing signal generator 80 generates a timing signalwhen the dimension of the white part of the eye has a larger dimensionthan a predetermined dimension. The width of the eye is always the same,even when the person opens or closes his/her eye. Therefore, thepredetermined dimension may be determined relative to the width of theeye. People usually blink both eyes at the same time, therefore, thetiming signal generator 80 may check only one of the eyes of thephotographed person. However, by checking both eyes, the desiredjudgement location can be selected more precisely.

[0126] When the photographing condition “the person's eyes are notred-eyed” is selected, the judgement location is the eyes and thereference situation is “the red part in his/her iris has a smalldimension”, as described above. Therefore, the timing signal generator80 calculates the dimension of the red part in the iris of the eyedetected by the judgement location detector 68 for the image, based onthe image information. The iris of his/her eye is recognized as being acylindrical or elliptic area whose circumference has a brownish orblue/green color. The timing signal generator 80 generates a timingsignal when the dimension of the red part of the eye has smallerdimension than a predetermined dimension. Both eyes of people areusually red eyed at the same time, therefore, the timing signalgenerator 80 may check only one of the eyes of the photographed person.However, by checking both of his/her eyes, the desired judgementlocation can be selected more precisely.

[0127] When the photographing condition “the person is looking at thecamera” is selected, the judgement location is the eye and the referencesituation is “the normal vector of the iris in his/her eye isapproximately equal to the angle of the line between the camera andhis/her iris”, as described above. Therefore, the timing signalgenerator 80 recognizes the iris as being a cylindrical or elliptic areawhose circumference has a brownish or blue/green color. The timingsignal generator 80 then recognizes the center of the iris and thenormal vector of the center of the iris. The timing signal generator 80generates a timing signal when the normal vector of the iris in the eyeis closer to the line between the camera and the iris than apredetermined distance.

[0128] The normal vector of the iris can be obtained from the relativeposition of the camera and the face of the person, the relative positionof the face and the eyes of the person, and the relative position of theeyes and the irises of the person. The timing signal generator 80 mayjudge the desired judgement location based on the normal vector obtainedfrom these relative positions.

[0129] When the photographing condition “the person is smiling” isselected, the judgement location is the eyes or the mouth and thereference situation is “the white part in his/her eye has a smalldimension”, “the width of his/her mouth is wide” or “the white part inhis/her mouth has a large dimension”, as described above. Therefore, thetiming signal generator 80 calculates the dimension of the white part ofthe eye, the width of the mouth, and the dimension of the white part ofthe mouth detected by the judgement location detector 68 for each of theimages, based on the image information. The timing signal generator 80generates a timing signal when the white part of the eye has a smallerdimension than a predetermined dimension, when the mouth has a widerwidth than a predetermined width, or when the white part of the mouthhas a larger dimension than a predetermined dimension. The predetermineddimension for the white part of the eye is relatively determined withrespect to the width of the eye. The predetermined width for the mouthis relatively determined with respect to the width of the face of theperson. The predetermined dimension for the white part of the mouth isrelatively determined with respect to the dimension of the face of theperson.

[0130] The timing signal generator 80 outputs a timing signal when thejudgement location satisfies the above reference situations. Asdescribed above, the control unit 50 extracts the face part based on theraw image and the information for the raw image. The control unit 50then detects the judgement location from the data for the extracted facepart. As the camera 10 photographs a subject when the detected judgementlocation satisfies the photographing condition, the camera 10 canautomatically photograph a desirable refined image without bothering thephotographer.

[0131] The method of generating a timing signal when a plurality ofpeople is photographed, will be explained next.

[0132] When each of the images includes a plurality of people, theextractor 60 extracts the aimed object and detects the judgementlocations for each of the people. This means that the aimed objectextractor 66 extracts the face parts for each of the people from each ofthe images. The judgement location extractor 68 detects the eyes or themouth for each of the people from each of the images.

[0133] When each of the images includes a plurality of people, thetiming signal generator 80 compares each of the judgement locations foreach of the people based on the reference situation for thephotographing condition. The timing signal generator 80 may generate atiming signal when the judgement locations for many of the peoplesatisfy the reference situation for the photographing condition. Thetiming signal generator 80 may output the timing signal when the ratioof the judgement locations satisfying the photographing conditionagainst all of the plurality of the judgement locations exceeds apredetermined ratio. In this case, the camera 10 can photograph arefined image in which many of the people have a good appearance.

[0134]FIG. 4 is a flowchart showing a method of photographing an image.The camera 10 starts photographing the subject when the release button52 is pushed (S100). When the camera 10 starts photographing, data for aparallactic image is input from the parallactic image data input unit 22(S102). At the same time, data for raw images are continuously inputfrom the image data input unit 24 (S104). Then, the aimed objectextractor 66 extracts the face part of the targeted person as the aimedobject (S106) . The judgement location detector 68 detects the judgementlocation based on the image information for the face part (S108). Thetiming signal generator 80 generates and outputs a timing signal whenthe judgement location satisfies a predetermined photographing condition(S110). Upon receiving the timing signal, the image pickup control unit56 controls the input unit 20 to photograph a refined image (S112).

[0135] The image-processing unit 84 processes the refined image, forexample, compositing images and the like (S114). The recording unit 90records the processed image on a recording medium (S116). The outputunit 92 outputs the recorded image (S118). The photographing operationis terminated (S120).

[0136]FIG. 5 is a flowchart showing in detail the method of extracting aface part, step 106 in FIG. 4. The depth information extractor 62extracts the depth information based on the parallactic image (S130).The image information extractor 64 extracts the image information basedon the raw image (Sl32). Then, the aimed object extractor 66 extractsthe face part of the targeted person based on the depth information andthe image information (S134). When each of the images includes aplurality of people, the aimed object extractor 66 extracts the faceparts for all of the people from each of the images (S136).

[0137]FIG. 6 is a flowchart showing in detail the method of detecting ajudgement location, step 108 in FIG. 4. The judgement location detector68 detects the judgement location based on the image information for theface part (S150). When each of the images includes a plurality ofpeople, the judgement location detector 68 detects the judgementlocations for all of the people (S152 and S150). Then, theinput-condition-determining unit 82 determines the input condition basedon the image information for the judgement location (S154).

[0138]FIG. 7 is a flowchart showing in detail the method of generating atiming signal, step 110 in FIG. 4. The timing signal generator 80 judgeswhether the judgement location detected by the judgement locationdetector 68 satisfies the photographing condition or not (S160). Thetiming signal generator 80 continues judging whether the judgementlocation satisfies the photographing condition or not for apredetermined period (S164 and S160). The timing signal generator 80generates a timing signal when the judgement location satisfies thephotographing condition (S162). The image pickup control unit 56controls the input unit 20 to stop photographing raw images when thejudgement location does not satisfy the predetermined photographingcondition for a predetermined period (S164 and S166).

[0139]FIG. 8 is a flowchart showing in detail the method ofphotographing a refined image, step 112 in FIG. 4. The image pickupcontrol unit 56 controls the input unit 20 to automatically photograph arefined image based on the timing signal output at the step 110 in FIG.4 (S170) . The input unit 20 inputs the data for the refined image(S172).

[0140] At the step 112 in FIG. 4, the camera 10 may not automaticallyphotograph a refined image but the user of the camera 10 may push therelease button 52 to photograph the refined image, upon receiving thealarm signal from the alarm 54.

[0141]FIG. 9 is a flowchart showing in detail the method ofphotographing a refined image, step 112 in FIG. 4. The alarm 54 outputsan alarm signal such as an alarm sound or an alarm light based on thetiming signal generated at the step 110 (Sl90). When the user, or thephotographer of the camera 10 notices the alarm signal, and then pushesthe release button 52 (S192), the camera 10 photographs a refined image(S194)

[0142] As the alarm 54 outputs the alarm sound or the alarm light basedon the timing signal, the user can photograph a refined image at anoptimum timing, without having to judge the timing himself. Furthermore,the targeted person can also notice the timing by the alarm sound or thealarm light.

[0143] The alarm 54 may output an alarm signal such as an alarm sound oran alarm light when the timing signal is not output from the timingsignal generator for a predetermined period.

[0144]FIG. 10 is a flowchart showing in detail the method of generatinga timing signal in which the alarm 54 outputs the alarm signal, step 110in FIG. 4. The timing signal generator 80 judges whether or not thejudgement location detected by the judgement location detector 68satisfies the photographing condition (S180). The timing signalgenerator 80 continues judging whether or not the judgement locationsatisfies the photographing condition for a predetermined period (S184and S180). The timing signal generator 80 generates a timing signal whenthe judgement location satisfies the photographing condition (S182). Thealarm 54 outputs an alarm signal such as the alarm sound and the alarmlight when the timing signal generator 80 does not output the timingsignal for a predetermined period (S184 and S186). The image pickupcontrol unit 56 controls the input unit 20 to stop photographing rawimages at this time (S188).

[0145] As the alarm 54 outputs an alarm signal such as an alarm soundand an alarm light when the timing signal is not output within apredetermined period, the photographer and the targeted person becomeaware of the fact that the targeted person does not meet thephotographing condition, by the sound and the light.

[0146]FIG. 11 shows a camera 110 of the second embodiment according tothe present invention. The camera 110 continuously photographs rawimages of a subject. The camera 110 then photographs a refined image ofthe subject, in accordance with a predetermined input condition, at thetiming when one of the previously photographed raw images satisfies apredetermined photographing condition. The camera 110 in this embodimentis a silver halide type camera by which an image of a subject is formedon a silver halide film. The camera 110 includes an input unit 120, anA/D converter30, a memory 40, a control unit 150, a release button 52and an alarm 54. The A/D converter 30, the memory 40, the release button52 and the alarm 54 in this embodiment have the same structures andfunctions as those explained in the first embodiment. Therefore, theexplanation of these parts will be omitted.

[0147] The input unit 120 includes a parallactic image data input unit122, a raw image data input unit 124 and a refined image data input unit130. The parallactic image data input unit 122 and the raw image datainput unit 124 in this embodiment respectively have the same structuresand functions as the parallactic image data input unit 22 and the imagedata input unit 24 explained in the first embodiment. The refined imagedata input unit 130 includes a lens 132, a lens stop 134, a shutter 136and a photographing unit 138. The lens 132, the lens stop 134 and theshutter 136 in this embodiment respectively have the same structures andfunctions as the lens 25, the lens stop 26 and the shutter 27 shown inFIG. 1 of the first embodiment. The photographing unit 138 receives anoptical image of a subject and forms an image of the subject on a silverhalide film.

[0148] The image data input unit 24 of the first embodiment inputs botha raw image and a refined image. As for the camera 110 of thisembodiment, the raw image data input unit 124 inputs an electronic rawimage and the refined image data input unit 130 inputs a refined imageand forms the refined image on a film. The raw image data input unit 124has a CCD for receiving the image of the subject in the same way as thedata input unit 24 of the first embodiment. The raw image data inputunit 124 outputs electronic signals for the image converted by the CCD.

[0149]FIG. 12 is a block diagram of the control unit 150 according tothe second embodiment. The control unit 150 includes an image pickupcontrol unit 56, an image forming control unit 58, an extractor 60, acondition-storing unit 70, a timing signal generator 80 and aninput-condition-determining unit 82. The extractor 60, thecondition-storing unit 70, the timing signal generator 80 and theinput-condition-determining unit 82 in this embodiment respectively havethe same structures and functions as those of the first embodiment, thusthe explanation of these parts will be omitted.

[0150] The image-forming control unit 58 controls the input unit 120 toform an image of a subject. The image forming control unit 58 controlsat least one of the following conditions of the input unit 120: focuscondition of the lens 132, aperture condition of the lens stop 134 andexposure time of the shutter 136, based on the input conditiondetermined by the input-condition-determining unit 82. The image-pickupcontrol unit 56 controls the input unit 120 to photograph an image of asubject. The image-pickup control unit 56 also controls thephotographing unit 138 to photograph a refined image, based on the inputcondition.

[0151] In this embodiment, the camera 110 includes the raw image datainput unit 124 for inputting an electronic raw image in addition to theimage data input unit 130 for inputting a refined image. Therefore, thecamera can automatically set an optimum condition for photographing arefined image of the subject. Thus, the desired refined image can beobtained without photographing a plurality of images using silver halidefilms, which can be expensive.

[0152] A camera of the third embodiment according to the presentinvention will be explained in the following. The camera of thisembodiment has the same structure as that of the first embodimentexplained with reference to FIG. 1. The camera of the third embodimentcontinuously photographs raw images of a subject. The camera thenphotographs a refined image, in accordance with a predetermined inputcondition, at the timing when the previously photographed raw imagesatisfies a predetermined photographing condition. The camera 110 mayhave a switch, not shown in the drawings, for selecting an automaticphotographing mode in which the best timing for photographing the imageis automatically determined, and a manual photographing mode in whichthe user of the camera 110 determines the best timing.

[0153] The camera of this embodiment has the same structure as that ofthe first embodiment and includes an input unit 20, an A/D converter 30,a memory 40, a control unit 50, a release button 52, an alarm 54, arecording unit 90 and an output unit 92. The camera of this embodimentmay be, for example, a digital still camera or a digital video camerathat can photograph a still image.

[0154]FIG. 13 is a block diagram of the control unit 50 according to thethird embodiment. The control unit 50 includes an image-pickup controlunit 56, an image-forming control unit 58, an extractor 60, acondition-storing unit 70, a photographing condition judging unit 80, aninput-condition-determining unit 82, and an image-processing unit 84.

[0155] The extractor 60 receives a parallactic image photographed by theparallactic image data input unit 22 and a normal image photographed bythe image data input unit 24, from the memory 40. The normal imageincludes a raw image and a refined image. The extractor 60 extracts anaimed object from the normal image based on the information obtainedfrom the parallactic image and the normal image. The informationincludes image information of the normal image and depth information ofthe parallactic image. The extractor 60 outputs data for the aimedobject to the input-condition-determining unit 82 and to theimage-processing unit 84.

[0156] As described above, cameras are usually used to photograph humanbeings. Therefore, the best timing for photographing a refined image maybe determined by the condition of a targeted person. Therefore, it isassumed that the extractor 60 extracts a face part of the targetedperson as the aimed object in this embodiment.

[0157] The extractor 60 then detects a judgement location from the aimedobject based on the information obtained from the parallactic images andthe normal images. It is also assumed that the extractor 60 detectsshapes or colors of the eyes or the mouth of the targeted person as thejudgement location in this embodiment.

[0158] The condition-storing unit 70 stores predetermined photographingconditions related to the judgement location, which should be includedin each of the raw images obtained by photographing the subject. Thecondition-storing unit 70 may store a plurality of photographingconditions. The condition-storing unit 70 may include acondition-setting unit, not shown in the drawings, by which a user canselect at least one of the photographing conditions from among aplurality of photographing conditions.

[0159] The best timing for photographing a refined image may be, forexample, the timing when the targeted person does a predeterminedmotion. This means that the best timing may be the timing when the aimedobject of the targeted person shows a predetermined variation. Thepredetermined variation may be, for example, “the person opens his/hereyes after he/she has been closing his/her eyes for more than twoseconds” or “the person's vision of sight follows a predeterminedtrail”. The condition storing unit 70 stores these conditions as thephotographing conditions.

[0160] The photographing condition judging unit 80 outputs a timingsignal for photographing an image. The photographing condition judgingunit 80 outputs the timing signal when the judgement location detectedby the extractor 60 shows a predetermined motion that satisfies thepredetermined photographing condition stored in the storing unit 70.

[0161] The input-condition-determining unit 82 determines an inputcondition for inputting an image based on the information for an aimedobject or the judgement location received from the extractor 60. Theinput-condition-determining unit 82 outputs the input condition to theimage forming control unit 58. The input condition may be, for example,focus condition of the lens 25 such that the aimed object including thejudgement location is focussed. As the input unit 20 inputs an image inaccordance with the input condition such as the focus condition of thelens 25, determined by the input-condition-determining unit 82, thecamera of this embodiment can photograph a refined image in which thesubject is in good condition.

[0162] The image-forming control unit 58 controls the input unit 20 toform a refined image of the subject based on the input conditiondetermined by the condition-determining unit 70. This means that theimage-forming control unit 58 controls at least one of the conditionsincluding focus condition of the lens 25, aperture condition of the lensstop 26, exposure time of the shutter 27, and condition of theparallactic shutter 34, based on the input condition.

[0163] The image pickup control unit 56 controls the input unit 20, tophotograph a refined image of the subject based on the input conditiondetermined by the condition-determining unit 70. This means that theimage-pickup control unit 56 controls at least one of the conditionsincluding output signal of the CCD 29 and output signal of theparallactic CCD 36, based on the input condition. The image-pickupcontrol unit 56 controls the input unit 20, to photograph a refinedimage based on the timing signal output from the photographing conditionjudging unit 80. The image-pickup control unit 56 controls theimage-processing unit 84 to process the refined image.

[0164] The image-processing unit 84 receives the refined imagephotographed by the image data input unit 24 from the memory 40. Theimage-processing unit 84 then processes the refined image based on theinformation for the aimed object or the judgement location extractedfrom the extractor 60. The refined image is processed in accordance withthe process conditions as explained in the first embodiment.

[0165]FIG. 14 is a functional block diagram of the extractor 60. Theextractor 60 includes a depth information extractor 62, an imageinformation extractor 64, an aimed object extractor 66 and a judgementlocation detector 68.

[0166] The depth information extractor 62 extracts the depth informationindicating the distance to each of components of the subject, based onthe data of the parallactic image received from the memory 40.

[0167] The image information extractor 64 extracts the image informationfor normal images, from the data for the normal images received from thememory 40. The image information includes, for example, data of thenormal image such as luminescence distribution, intensity distribution,color distribution, texture distribution, and motion distribution.

[0168] The aimed object extractor 66 extracts data for the face area ofthe person as the aimed object, based on the depth information and theimage information. The aimed object is extracted in a similar manner asthat explained in the first embodiment.

[0169] The aimed object extractor 66 outputs the information for theaimed object to the input-condition-determining unit 82 and theimage-processing unit 84.

[0170] As described above, the aimed object extractor 66 extracts anaimed object based on the depth information in addition to the imageinformation. Therefore, even when a plurality of people are photographedin the image and their faces are close to each other, the faces of thedifferent people can be distinctly extracted.

[0171] The judgement location detector 68 detects the judgement locationfrom the data for the aimed object extracted by the aimed objectextractor 66. The judgement location is detected in accordance with adetecting condition different from the extracting condition forextracting the aimed object by the aimed object extractor 66. In thisembodiment, the judgement location is eyes or mouth of the photographedperson. Therefore, the judgement location detector 68 detects the eyesand mouth from the face area.

[0172] The judgement location detector 68 outputs the information forthe judgement location to the photographing condition judging unit 80.

[0173]FIG. 15 is a block diagram of the function of the photographingcondition judging unit 80. The photographing condition judging unit 80includes a detection-starting unit 85, a variation detector 86 and ajudging unit 88. The photographing condition includes a predeterminedphotographing condition related to the motion of the judgement locationof the aimed object, and the starting condition for starting detectionof the motion of the judgement location.

[0174] The detection-starting unit 85 outputs a starting signal when thejudgement location detected by the extractor 60 satisfies apredetermined starting condition. The variation detector 86 startsdetecting variation in the motion of the judgement location uponreceiving the starting signal from the detection-starting unit 85. Thejudging unit 88 outputs the timing signal for photographing a refinedimage when the variation of the motion of the judgement locationdetected by the variation detector 86 satisfies a predeterminedphotographing condition.

[0175] The photographing conditions may be, for example, “the personopens his/her eyes after he/she has been closing his/her eyes for morethan two seconds” or “the person's vision of sight follows apredetermined trail”. It is desirable that the photographing conditionsare motions or variations of the targeted person, which the targetedperson usually does not perform in front of the camera, in order toavoid misjudgment.

[0176] Each of the photographing conditions has a reference situationfor the judgement location, which should meet the requirements of thephotographing condition. The condition-storing unit 70 also stores thereference situations for the judgement location, each respectivelycorresponding to each of the photographing conditions. The referencesituations for the judgement location corresponding to each of thephotographing conditions will be described in the following.

[0177] For the conditions such as “the person is not blinking”, and “theperson is looking at the camera”, the reference situation may relate toshape of the eye, color of the eye, and size of the eye. Whether each ofthe judgement locations satisfies each of these reference situations ornot is judged in accordance with predetermined algorithms based onexperience.

[0178] When the photographing condition “the person opens his/her eyesafter he/she has been closing his/her eyes for more than two seconds” isselected, the judgement location may be the eye of the person. Thereference situation for the eye in this photographing condition will bedetermined as follows. When a person blinks, his/her eyelid hideshis/her eyeball. While he/she is blinking and his/her eye is partiallyclosed, a white part of his/her eyeball is especially hidden by his/hereyelid. This means that when the person is blinking, the white part ofhis/her eyeball should be relatively small and when the person is notblinking, the white part of his/her eyeball should be relatively large.Therefore, whether the person opens his/her eyes or not is determinedbased on the dimension of the white part of his/her eyeball.

[0179] The starting condition for the photographing condition “theperson opens his/her eyes after he/she has been closing his/her eyes formore than two seconds” becomes “the person closes his/her eyes”. Thedetection-starting unit 85 outputs a starting signal when it detects theclosed eye of the person. The variation detector 86 starts detectingvariation of the eye upon receiving the starting signal. The variationdetector 86 counts the period while the person keeps his/her eyesclosed, from the data for the raw images continuously input. Thevariation detector 86 outputs the timing signal when the person openshis/her eyes after he/she has had his/her eyes closed for more than twoseconds. It is desirable for the variation detector 86 to output thetiming signal one second after the person opens his/her eyes rather thanthe moment when the person opens his/her eyes.

[0180] When the photographing condition “the person's vision of sightfollows a predetermined trail” is selected, the judgement location maybe the eye of the person. The reference situation for the eye in thisphotographing condition will be determined as follows. The trail of theperson's vision of sight can be detected by detecting the normal vectorof the iris in the eye. At first, the iris of his/her eye is recognized,from his/her eye detected by the judgement location detector 68, asbeing a cylindrical or elliptic area whose circumference has a brownishor blue/green color. The center of the iris is then detected based onthe image information for the eye. The normal vector of the center ofthe iris is obtained based on the depth information.

[0181] It is assumed in this photographing condition that thepredetermined trail of the vision of sight is “the person looks upperleft with respect to the camera, lower right with respect to the cameraand then at the camera”, for example. The starting condition in thiscase becomes “the person looks to the upper left with respect to thecamera”. The detection starting unit 85 outputs a starting signal whenit detects that the person is looking to the upper left with respect tothe camera. The variation detector 86 starts detecting variation of thevision of sight of the person upon receiving the starting signal. Thevariation detector 86 detects the trail of the vision of sight based onthe data for the plurality of input raw images. The judgement unit 88outputs the timing signal when the trail is “upper left, lower right andthen at the camera”.

[0182] The control unit 50 extracts the face part based on the data forthe raw image and the information thereof and then detects the judgementlocation from the information for the extracted face part. The controlunit 50 then detects the variation of the judgement location anddetermines the timing for photographing when the detected judgementlocation satisfies the photographing condition. Therefore, the camera ofthis embodiment can automatically photograph at a timing when thetargeted person is in good condition.

[0183] The method of generating a timing signal when a plurality ofpeople is photographed will be explained next.

[0184] When each of the images includes a plurality of people, thejudgement location detector 68 detects the judgement locations for eachof the people. This means that the aimed object extractor 66 extractsthe face parts for each of the people from each of the images. Thejudgement location extractor 68 detects the eyes or the mouth for eachof the people from each of the images.

[0185] At this time, the variation detector 86 detects the variation ofthe judgement locations for each of the people. The judging unit 88outputs the timing signal when the variation of the plurality ofjudgement locations satisfy the photographing condition. The judgingunit 88 selects the aimed objects respectively, including the judgementlocations whose variation satisfies the photographing condition. Thejudging unit 88 then outputs the information of the aimed objectsincluding the selected judgement locations to theinput-condition-determining unit 82 and the image-processing unit 84.

[0186] The method of photographing an image in this embodiment is almostsame as that of the first embodiment shown in FIGS. 4 and 5.

[0187]FIG. 16 is a flowchart showing in detail the method of detecting ajudgement location, step 108 in FIG. 4. The judgement location detector68 detects the judgement location based on the image information for theface part (S250). When each of the images includes a plurality ofpeople, the judgement location detector 68 detects the judgementlocations for all of the people (S252 and S250).

[0188]FIG. 17 is a flowchart showing in detail the method of generatinga timing signal, step 110 in FIG. 4. The detection starting unit 85judges whether or not the judgement location detected by the judgementlocation detector 68 satisfies the starting condition (S260). Thedetection-starting unit 85 continues judging whether or not thejudgement location satisfies the starting condition for a predeterminedperiod (S260 and S262). The variation detector 86 starts detecting thevariation of the judgement location when the judgement locationsatisfies the starting condition (S261). The image-pickup control unit56 controls the input unit 20 to stop photographing raw images when thejudgement location does not satisfy the predetermined starting conditionfor a predetermined period (S262 and S263).

[0189] The judging unit 88 then judges whether the variation of thejudgement location satisfies the photographing condition or not (S264).The timing signal generator 80 generates a timing signal when thevariation of the judgement location satisfies the photographingcondition (S265). When the variation of the judgement location does notsatisfy the photographing condition, the process returns to step S260 ifthe predetermined period is remaining. Then, the detection starting unit85 judges again whether or not the judgement location detected by thejudgement location detector 68 satisfies the starting condition (S260).The image pickup control unit 56 controls the input unit 20 to stopphotographing raw images when the predetermined period is expired (S266and S267).

[0190]FIG. 18 is a flowchart showing in detail the method ofphotographing a refined image, step 112 in FIG. 4. The image-pickupcontrol unit 56 controls the input unit 20 to automatically photograph arefined image based on the timing signal output at the step 110 in FIG.4 (S270). The input unit 20 inputs the data for the refined image(S272).

[0191] At the step 112 in FIG. 4, the camera 10 may not automaticallyphotograph a refined image but the user of the camera 10 may press therelease button 52 to photographing the refined image upon receiving thealarm signal from the alarm 54.

[0192] The method of manually photographing a refined image by the userof the camera 10 is in accordance with the flowchart shown in FIG. 9,which is explained in the first embodiment. The alarm 54 outputs analarm signal such as an alarm sound or an alarm light based on thetiming signal generated at the step 110 (S190) . When the user, or thephotographer of the camera 10 notices the alarm signal, and then he/shepresses the release button 52 (S192), the camera 10 photographs arefined image (S194).

[0193] As the alarm 54 outputs the alarm sound or the alarm light basedon the timing signal, the user can photograph a refined image at anoptimum timing without having to judge the timing himself. Furthermore,the targeted person can also notice the timing because of the alarmsound or the alarm light.

[0194] The alarm 54 may output an alarm signal such as an alarm sound oran alarm light when the timing signal is not output from the timingsignal generator for a predetermined period.

[0195]FIG. 19 is a flowchart showing in detail the method of generatinga timing signal in which the alarm 54 outputs the alarm signal, step 110in FIG. 4. The detection-starting unit 85 judges whether or not thejudgement location detected by the judgement location detector 68satisfies the starting condition (S300). The detection starting unit 85continues judging whether or not the judgement location satisfies thestarting condition for a predetermined period (S300 and S304). Thevariation detector 86 starts detecting the variation of the judgementlocation when the judgement location satisfies the starting condition(S302). The alarm 54 outputs an alarm signal such as an alarm sound andan alarm light when the photographing condition judging unit 80 does notoutput the timing signal for a predetermined period (S304 and S306).Then, the image-pickup control unit 56 controls the input unit 20 tostop photographing raw images, when the judgement location does notsatisfy the predetermined starting condition for a predetermined period(S308).

[0196] The judging unit 88 then judges whether or not the variation ofthe judgement location satisfies the photographing condition (S310). Thetiming signal generator 80 generates a timing signal when the variationof the judgement location satisfies the photographing condition (S312).When the variation of the judgement location does not satisfy thephotographing condition, the process returns to the step S314 if thepredetermined period is not remaining. Then, the detection starting unit85 judges again whether or not the judgement location detected by thejudgement location detector 68 satisfies the starting condition (S314and S300). At the step S314, the alarm 54 outputs an alarm signal suchas an alarm sound and an alarm light when the predetermined period isexpired (S316). The image-pickup control unit 56 controls the input unit20 to stop photographing raw images (S318).

[0197] As the alarm 54 outputs the alarm signal such as the alarm soundand the alarm light when the timing signal is not output within apredetermined period, the photographer and the targeted person becomeaware of the fact that the targeted person does not meet thephotographing condition because of the sound and the light.

[0198] The camera of the fourth embodiment will be explained in thefollowing. The camera of this embodiment is a silver halide type cameraby which an image of a subject is formed on a silver halide film and hasthe same structure as that explained in the second embodiment shown inFIG. 11. Therefore, the explanation of the structure of the camera inthis embodiment will be omitted.

[0199]FIG. 20 is a block diagram of the control unit 150 in thisembodiment. The control unit 150 in this embodiment includes an imagepickup control unit 56, an image forming control unit 58, an extractor60, a condition storing unit 70, a photographing condition judging unit180, an input-condition-determining unit 82. The extractor 60, thecondition storing unit 70, the photographing condition judging unit 180and the input-condition-determining unit 82 in this embodimentrespectively have same the structures and functions as those explainedin the first embodiment, therefore, the explanation of these parts willbe omitted.

[0200] The image-forming control unit 58 controls the input unit 120 toform an image of a subject. The image-forming control unit 58 controlsat least one of the following conditions of the input unit 120: focuscondition of the lens 132, aperture condition of the lens stop 134 andexposure time of the shutter 136, based on the input conditiondetermined by the input-condition-determining unit 82. The image pickupcontrol unit 56 controls the input unit 120 to photograph an image of asubject. The image pickup control unit 56 also controls thephotographing unit 138 to photograph a refined image based on the inputcondition.

[0201] In this embodiment, the camera 110 includes the raw image datainput unit 124 for inputting an electronic raw image, in addition to theimage data input unit 130 for inputting a refined image. Therefore, thecamera can automatically set an optimum condition for photographing arefined image of the subject. Thus, a desired refined image can beobtained without photographing a plurality of images using silver halidefilms, which can be expensive.

[0202] A camera of the fifth embodiment according to the presentinvention will be explained in the following. The camera of thisembodiment continuously photographs images of a subject. The cameraoutputs a timing signal when the targeted subject in the image satisfiesthe photographing condition. Upon receiving the timing signal, thecamera of this embodiment records one of the images, which wasphotographed at a predetermined earlier period than the timing signal,based on the timing signal, as a refined image.

[0203] The camera of this embodiment includes a control unit 50. Thestructure of the camera of this embodiment other than the control unit50 is the same as that explained in the first to fourth embodiments.Thus, the explanation of same parts will be omitted.

[0204]FIG. 21 is a block diagram of the control unit 50 according to thefourth embodiment. The control unit 50 includes an extractor 60, acondition-storing unit 70, a timing signal generator 80, an imageprocessing unit 84, and a image storing unit 140. The extractor 60, thecondition-storing unit 70, the timing signal generator 80 and the imageprocessing unit 84 are the same as those explained in the first tofourth embodiments. Although only the timing signal generator 80 isshown in FIG. 21, the part having the numeral 80 may be thephotographing condition judging unit explained in the third and thefourth embodiments.

[0205] The image storing unit 140 temporarily stores the imagesphotographed by the image data input unit 24 and input from the memory40. Each of the images is respectively stored with time records of whenthe image was photographed. The image storing unit 140 receives thetiming signal from the timing signal generator 80 and then outputs oneof the raw images photographed at a timing earlier than the timingsignal by a predetermined period as the refined image, to the imageprocessing unit 84. The image processing unit 84 processes the refinedimage based on the information for the extractor 60.

[0206]FIG. 22 is a flowchart showing a method of photographing an image.The camera starts photographing the subject when the release button 52is pressed (S400) . When the camera starts photographing, data for aparallactic image is input from the parallactic image data input unit 22(S402). At the same time, data for raw images are continuously inputfrom the image data input unit 24 (S404). The raw images are temporarilystored in the image storing unit 140. Then, the aimed object extractor66 extracts the face part of the targeted person as the aimed object(S406). The judgement location detector 68 detects the judgementlocation based on the image information for the face part (S408). Thephotographing condition judging unit 180 generates and outputs a timingsignal when the judgement location satisfies a predeterminedphotographing condition (S410). Upon receiving the timing signal, theimage storing unit 140 selects one of the raw images photographed at atiming earlier than the timing signal by a predetermined period, as therefined image. The image storing unit 140 outputs the refined image tothe image-processing unit 84 (S412).

[0207] The image-processing unit 84 processes the refined image (S414).The processing of the refined image may include compositing a pluralityof refined images and the like. The recording unit 90 records theprocessed image on a recording medium (S416). The output unit 92 outputsthe processed image (S418), and the photographing operation isterminated (S420).

[0208] The detailed operations of the steps 206, 208 and 210 are thesame as those explained in the previous embodiments. Thus, anexplanation of these steps will be omitted.

[0209] The image storing unit 140 may store all of the raw images whichare photographed from a timing earlier than the timing signal by apredetermined period to the timing of the timing signal, as the refinedimages. In this case, the image-processing unit 84 processes theplurality of refined images.

[0210] As described above, the camera stores the raw image which isphotographed at a timing earlier than the timing signal by apredetermined period as the refined image, based on the timing signal.Therefore, the refined image is selected by considering the delay time,even when the extractor 60 takes a certain time for extracting the aimedobject and detecting the judgement location. Thus, an image in which thetargeted person has a good appearance can be obtained.

[0211] Furthermore, the camera stores all of the raw images which arephotographed from a timing earlier than the timing signal by apredetermined period to the timing of the timing signal, as the refinedimages. Therefore, an image in which the targeted person has a goodappearance can be selected.

[0212]FIG. 23 shows a camera 210 of the sixth embodiment according tothe present invention. The camera 210 of this embodiment continuouslyphotographs a plurality of raw images of a subject in the same way asthe first to fifth embodiments. The camera 210 outputs a timing signalwhen the raw image satisfies the photographing condition.

[0213] The camera 210 of this embodiment has the same structure as thatof the first embodiment and further includes a communication unit 150.The camera 210 outputs the timing signal through the communication unit150, to control operation of an external apparatus 160 based on thetiming signal. The communication unit 150 of the camera 210 sends thetiming signal to the external apparatus 160 by a wireless means. Thecommunication unit 150 of the camera 210 and the external apparatus maybe held in communication with each other by a wireless means such as viaa radio or infrared radiation or by cables such as via a USB or a LAN.The external apparatus 160 may be, for example, a camera forphotographing a refined image of the target, or an illuminator.

[0214] In this embodiment, the camera 210 continuously photographs rawimages of a subject. The camera 210 outputs a timing signal when the rawimage satisfies a predetermined selecting condition. The timing signalis transferred from the camera 210 to the external apparatus 160 throughthe communication unit 150 of the camera 210. When the externalapparatus 160 is another camera for photographing a refined image, theexternal apparatus photographs a refined image of the subject based onthe timing signal from the camera 210.

[0215] Using the camera 210 of this embodiment, even a silver halidetype camera that does not include a raw image data input unit canphotograph a refined image of a subject at the timing when the targetedperson is in good condition. Thus, a desired refined image can beobtained without photographing a plurality of images using silver halidefilms which can be expensive.

[0216] As described above, according to the embodiments of the presentinvention, an image in which a targeted object satisfies a predeterminedphotographing condition.

[0217] Although the present invention has been described by way ofexemplary embodiments, it should be understood that many changes andsubstitutions may be made by those skilled in the art without departingfrom the spirit and the scope of the present invention which is definedonly by the appended claims.

What is claimed is:
 1. A camera comprising: an image data input unitforming a plurality of images of a subject for photographing saidsubject; a condition storing unit storing a predetermined photographingcondition related to a desirable variation of said subject; a variationdetector detecting variation of said subject in said plurality of saidimages based on information of said plurality of images; and a timingsignal generator outputting a timing signal when said variation of saidsubject satisfies said photographing condition.
 2. A camera as set forthin claim 1, further comprising: an extractor extracting data of an aimedobject from each of said plurality of images of said subject based on anextracting condition, wherein said photographing condition includes apredetermined condition related to a desirable aimed object, saidvariation detector detects variation of said aimed object in saidplurality of images based on said information of said plurality ofimages, and said timing signal generator outputs said timing signal whensaid variation of said aimed object satisfies said photographingcondition.
 3. A camera as set forth in claim 2, wherein said extractingcondition is based on depth information of said plurality of imagesindicating the distance to each part of said subject.
 4. A camera as setforth in claim 2, wherein said extractor detects data of a judgementlocation from said data of said aimed object in each of said pluralityof images based on a detecting condition different from said extractingcondition, said photographing condition includes a predeterminedphotographing condition related to a desirable judgement location, saidvariation detector detects variation of said judgement location in saidplurality of images based on said information of said plurality ofimages, and said timing signal generator outputs said timing signal whensaid variation of said judgement location satisfies said photographingcondition.
 5. A camera as set forth in claim 4, wherein saidphotographing condition includes a predetermined starting condition forstarting detection of said variation of said judgement location, andsaid variation detector starts detecting said variation of saidjudgement location when said judgement location satisfies said startingcondition.
 6. A camera as set forth in claim 2, wherein said extractorextracts data of a plurality of said aimed objects from each of saidplurality of images, said variation detector detects variation of eachof said plurality of said aimed objects in said plurality of imagesbased on information of said plurality of images, and said timing signalgenerator outputs said timing signal when said variation of saidplurality of said aimed objects satisfy said photographing condition. 7.A camera as set forth in claim 6, wherein said extractor detects data ofa plurality of judgement locations from each of said data of saidplurality of aimed objects based on a detecting condition different fromsaid extracting condition, said photographing condition includes apredetermined photographing condition related to desirable variation ofsaid judgement location, said variation detector detects variation ofeach of said plurality of said judgement locations in said plurality ofimages based on information of said plurality of images, and said timingsignal generator outputs said timing signal when said variation of saidplurality of said judgement locations satisfy said photographingcondition.
 8. A camera as set forth in claim 1 further comprising animage pickup control unit controlling said input unit for photographingsaid image based on said timing signal.
 9. A camera as set forth inclaim 1, further comprising an illuminator illuminating said subjectbased on said timing signal.
 10. A camera as set forth in claim 1,further comprising a recording unit recording said image on areplaceable nonvolatile recording medium based on said timing signal.11. A camera as set forth in claim 1, further comprising an alarmoutputting an alarm signal for notifying that said subject satisfiessaid photographing condition based on said timing signal.
 12. A cameraas set forth in claim 1, wherein said photographing condition includes aplurality of photographing conditions, and said camera further comprisesa condition-setting unit previously selecting at least one of saidphotographing conditions for photographing said image, from among saidplurality of photographing conditions.
 13. A camera as set forth inclaim 8, wherein said timing signal generator selects said judgementlocation satisfying said photographing condition from among saidplurality of said judgement locations in said plurality of images, andoutputs information for said aimed object including said judgementlocation, and said camera further comprising: an input conditiondetermining unit determining an input condition for inputting said imagebased on information for said judgement location; and an image formingcontrol unit controlling an input unit for forming said image of saidsubject based on said input condition.
 14. A camera as set forth inclaim 8, wherein said timing signal generator selects said judgementlocation satisfying said photographing condition from among saidplurality of said judgement locations in said plurality of images, andoutputs information for said aimed object including said judgementlocation, and said camera further comprising an image processing unitprocessing said image based on said information for said judgementlocation.
 15. A method of photographing a plurality of images of asubject comprising: detecting variation of said subject in saidplurality of said images based on information for said plurality ofimages; outputting a timing signal when said variation of said subjectsatisfies a predetermined photographing condition related to a desirablevariation of said subject.
 16. A method as set forth in claim 15,further comprising extracting data of an aimed object from each of saidplurality of images of said subject based on an extracting condition,said detecting includes detecting variation of said aimed object basedon information for said image, and said timing signal is output whensaid variation of said aimed object satisfies said photographingcondition.
 17. A method as set forth in claim 16, wherein saidextraction of said aimed object includes detecting data of a judgementlocation from said data of said aimed object in each of said pluralityof images based on a detecting condition different from said extractingcondition, said detecting variation of said subject includes detectingvariation of said judgement location based on information for saidimage, and said timing signal is output when said variation of saidjudgement location satisfies said photographing condition.
 18. A methodas set forth in claim 17, wherein said photographing condition includesa predetermined starting condition for starting detection of saidvariation of said judgement location, and said detecting of variationstarts detecting said variation of said judgement location when saidjudgement location satisfies said starting condition.
 19. A method asset forth in claim 15, further comprising photographing said image basedon said timing signal.